Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[network] Remove Legacy Gossiper + Only Push Gossip from RPC #497

Merged
merged 47 commits into from
Feb 29, 2024

Conversation

patrick-ogrady
Copy link
Contributor

@patrick-ogrady patrick-ogrady commented Feb 26, 2024

Related: ava-labs/avalanchego#2772

This PR removes the legacy tx gossiper and switches to exclusively use the AvalancheGo PushGossiper (https://github.com/ava-labs/avalanchego/blob/master/network/p2p/gossip/gossip.go). Additionally, this PR updates coreth to only send push gossip for transactions submitted over RPC.

TODO

  • restore commented out tests
  • all tests passing
  • ensure all artifacts from old gossip handling are cleaned up (ex: regossip stats)
  • Add submitted txs to bloom filter to avoid getting gossiped back what we gossiped Already handled:

    coreth/plugin/evm/gossip.go

    Lines 133 to 165 in 9d9639b

    func (g *GossipEthTxPool) Subscribe(ctx context.Context) {
    g.mempool.SubscribeNewTxsEvent(g.pendingTxs)
    for {
    select {
    case <-ctx.Done():
    log.Debug("shutting down subscription")
    return
    case pendingTxs := <-g.pendingTxs:
    g.lock.Lock()
    optimalElements := (g.mempool.PendingSize(false) + len(pendingTxs.Txs)) * txGossipBloomChurnMultiplier
    for _, pendingTx := range pendingTxs.Txs {
    tx := &GossipEthTx{Tx: pendingTx}
    g.bloom.Add(tx)
    reset, err := gossip.ResetBloomFilterIfNeeded(g.bloom, optimalElements)
    if err != nil {
    log.Error("failed to reset bloom filter", "err", err)
    continue
    }
    if reset {
    log.Debug("resetting bloom filter", "reason", "reached max filled ratio")
    g.mempool.IteratePending(func(tx *types.Transaction) bool {
    g.bloom.Add(&GossipEthTx{Tx: tx})
    return true
    })
    }
    }
    g.lock.Unlock()
    }
    }
    }
  • Handle the case where we restore local transactions from the journal (I don't think this is currently handled correctly for push gossip anyways, so we may want to punt to when we actually allow people to use the journal):
    // If local transactions and journaling is enabled, load from disk
    if !config.NoLocals && config.Journal != "" {
    pool.journal = newTxJournal(config.Journal)
    if err := pool.journal.load(pool.AddLocals); err != nil {
    log.Warn("Failed to load transaction journal", "err", err)
    }
    if err := pool.journal.rotate(pool.local()); err != nil {
    log.Warn("Failed to rotate transaction journal", "err", err)
    }
    }
    (Remove unused journaling configs #500)
  • Consider adding an "executable" check to the mempool so we only gossip txs that can be used (i.e. don't gossip txs with nonces) -> may not work great under load because peers won't be given a "run" of transactions to include If we take a pessimistic view on our ability to determine the usefulness of any particular transaction, what we do now is the most robust approach.

eth/api_backend.go Outdated Show resolved Hide resolved
plugin/evm/vm.go Outdated Show resolved Hide resolved
eth/backend.go Outdated Show resolved Hide resolved
@patrick-ogrady patrick-ogrady changed the title [network] Gossip Cleanup [network] Remove Legacy Gossiper + Only Push Gossip from RPC Feb 27, 2024
@aaronbuchwald
Copy link
Collaborator

aaronbuchwald commented Feb 27, 2024

If we're going to make this change, we may want to enable LocalTxs by default as well. Otherwise, this will break anyone who is issuing transactions to their own node (using the defaults) and relying on remote gossip to gossip it to the network.

Ignore this, I misunderstood. I thought the change was making it so we only push gossip locals (since we mark transactions as local when they're issued via the API), but we explicitly push gossip transactions if they are added via API instead of using the tx pool's definition of local.

Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
Copy link
Collaborator

@ceyonur ceyonur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just few nits

eth/api_backend.go Show resolved Hide resolved
plugin/evm/vm.go Show resolved Hide resolved
@patrick-ogrady
Copy link
Contributor Author

patrick-ogrady commented Feb 29, 2024

Failing test:

panic: test timed out after 30m0s
running tests:
	TestEthTxPushGossipInbound (29m23s)
goroutine 10559 [semacquire, 29 minutes]:
sync.runtime_Semacquire(0xb11a20?)
	/opt/hostedtoolcache/go/1.21.7/x64/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc008effd58?)
	/opt/hostedtoolcache/go/1.21.7/x64/src/sync/waitgroup.go:116 +0x48
github.com/ava-labs/coreth/core/txpool.(*TxPool).Stop(0xc007666000)
	/home/runner/work/coreth/coreth/core/txpool/txpool.go:464 +0x68
github.com/ava-labs/coreth/eth.(*Ethereum).Stop(0xc006b0d1d0)
	/home/runner/work/coreth/coreth/eth/backend.go:364 +0x45
github.com/ava-labs/coreth/plugin/evm.(*VM).Shutdown(0xc005f73800, {0xc00810c000, 0xc00810c000})
	/home/runner/work/coreth/coreth/plugin/evm/vm.go:1271 +0x185
github.com/ava-labs/coreth/plugin/evm.TestEthTxPushGossipInbound.func1()
	/home/runner/work/coreth/coreth/plugin/evm/tx_gossip_test.go:390 +0x28
github.com/ava-labs/coreth/plugin/evm.TestEthTxPushGossipInbound(0xc00884a300?)
	/home/runner/work/coreth/coreth/plugin/evm/tx_gossip_test.go:415 +0x9e6
testing.tRunner(0xc00810c000, 0x189eac0)
	/opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1595 +0xff
created by testing.(*T).Run in goroutine 1
	/opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x3ad

@StephenButtolph StephenButtolph merged commit 35e21ab into master Feb 29, 2024
8 checks passed
@StephenButtolph StephenButtolph deleted the gossip-cleanup branch February 29, 2024 01:36
oxbee pushed a commit to taurusgroup/coreth that referenced this pull request Nov 6, 2024
…s#497)

Co-authored-by: Joshua Kim <[email protected]>
Co-authored-by: Stephen Buttolph <[email protected]>
Co-authored-by: Darioush Jalali <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

6 participants